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on  the  connectivity  of  the  resultant  state- dependent  graph.  We  present  some 
simulations  to  validate  the  proposed  control  scheme.  Finally,  we  address  the 
problem  of  jamming  for  the  scenario  in  which  each  agent  computes  its  con¬ 
trol  strategy  based  on  limited  information  available  about  its  neighbors  in  the 
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1.  Introduction. 

An  object  in  possession  seldom  retains  the  same  charm  that  it  had 

in  pursuit. 

-Galius  Plinius  Secundus 

Situations  involving  pursuit  and  evasion  frequently  arise  in  combat  scenarios. 
Game  theory,  especially  differential  games,  is  an  important  tool  in  analyzing  such 
situations  of  conflict.  One  of  the  earliest  works  that  illustrates  the  connection 
between  differential  games  and  pursuit-evasion  is  the  seminal  work  of  Isaacs  that 
culminated  in  his  book  [27].  A  general  framework  based  on  the  concepts  in  classical 
game  theory  and  the  notion  of  tenet  of  transition  was  used  to  analyze  pursuit-evasion 
problems.  Classical  problems  like  the  Lion  and  the  Man^  Homicidal  Chauffeur  and 
Maritime  Dogfight  were  introduced  in  this  book.  In  addition  to  the  formulation 
of  these  problems  that  relate  to  real-life  scenarios,  Isaacs’  book  also  provides  the 
necessary  conditions  for  optimal  trajectories  for  the  players.  In  1967,  within  two 
years  after  the  publication  of  Rufus  Isaacs’  book  on  differential  games,  Leitmann 
developed  an  analysis  similar  to  that  of  Isaacs  in  [15],  by  building  on  his  own 
geometric  approach  to  control  theory  [34].  He  based  his  geometric  viewpoints  on 
the  properties  of  certain  surfaces  which  contain  the  optimal  trajectories:  the  limiting 
surfaces  in  control  theory  and  the  game  surfaces  in  differential  games.  An  elaborate 
history  of  the  various  generalizations  and  modifications  of  the  classical  problems  in 
pursuit-evasion  and  the  formulation  of  new  problems,  is  presented  in  [5]  and  [40]. 
In  the  past,  most  of  the  problems  in  pursuit-evasion  games,  for  example  [33],  [35], 
[38]  and  [37]  have  tried  to  relate  the  performance  of  the  players  to  their  agility  and 
on-board  energy.  In  this  paper,  we  investigate  a  novel  pursuit-evasion  game  that 
relates  the  agility  of  the  agents  in  a  team  to  their  capability  to  maintain  a  secure 
communication  network  in  the  presence  of  an  adversary. 

The  problem  under  consideration  in  this  paper  is  inspired  by  recent  discoveries 
of  jamming  instances  in  biological  species  [63].  Jamming  is  a  malicious  attack 
whose  objective  is  to  disrupt  the  communication  of  the  victim  network  intentionally, 
causing  interference  or  collision  at  the  receiver  side.  Jamming  attack  is  a  well- 
studied  and  an  active  area  of  research  in  wireless  networks.  [9]  provides  a  list  of 
references  on  wireless  networks  related  to  the  problem  considered  in  this  paper.  In 
contradistinction  to  the  contemporary  research  on  jamming  which  mainly  deals  with 
spectral  techniques  to  initiate  as  well  as  mitigate  jamming,  our  current  work  focuses 
on  mobility  as  a  means  to  evade  the  jamming  attack  within  the  communication 
network  present  among  the  agents  in  a  team.  The  effectiveness  of  a  jamming  attack 
depends  on  the  vicinity  of  the  transmitter  and  receiver  from  the  jammer.  This  gives 
rise  to  various  scenarios  involving  pursuit-evasion  games  for  a  mobile  jammer  in  the 
presence  of  a  mobile  transmitter  and  receiver.  Moreover,  our  previous  work  in  [13] 
highlights  the  fact  that  optimal  motion  strategies  for  the  receiver  and  transmitter 
to  evade  a  jamming  attack  are  difficult  to  compute  in  real  time.  This  work  is  in 
the  same  vein  with  an  additional  emphasis  on  efficient  approximation  techniques 
that  maintain  a  balance  between  the  complexity  and  the  optimality  of  the  resultant 
strategies  for  the  agents  in  the  following  scenarios;  large  team  size,  and  partial  state 
information  for  each  agent. 

The  main  contributions  of  this  paper,  based  on  our  prior  work  in  [9,  10,  7,  25], 
are  as  follows.  First,  we  use  a  differential  game  theoretic  approach  to  analyze  the 
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jamming  evasion  problem  in  mobile  networks,  which  has  never  been  seen  in  the 
pursuit-evasion  literature,  and  provides  a  novel  application  domain  of  significant 
interest.  Second,  we  extend  the  current  work  on  connectivity  maintenance  problems 
in  multi-agent  systems  to  include  antagonistic  scenarios  that  might  arise  due  to 
the  presence  of  an  adversary  in  the  vicinity.  Third,  we  address  the  problem  of 
decentralized  decision  making  in  adversarial  situations  in  context  of  the  jamming 
problem,  which  is  a  topic  of  interest  to  researchers  in  the  areas  of  games,  control 
and  decision-making. 

nodes  we  used  the  notion  of  state- dependent  graphs  to  model  the  problem.  Ap¬ 
plying  tools  from  algebraic  graph  theory  on  the  state-dependent  graphs  provided 
us  with  locally  optimal  control  strategies  for  the  agents  as  well  as  the  jammer. 
Finally,  we  proposed  two  approximation  techniques  for  the  agents  in  the  face  of  de¬ 
centralized  information  pattern.  First,  we  assumed  that  each  agent  has  a  knowledge 
about  the  value  function  under  perfect  state  information,  beforehand.  Due  to  lack 
of  information  about  all  the  agents  in  the  team,  each  agent  is  constrained  to  make 
a  local  decision  based  on  the  information  about  his  neighbors.  We  proposed  an 
online  algorithm  under  decentralized  information  pattern  for  convergence  for  each 
player.  We  proposed  an  approximation  scheme  for  the  agents  based  on  averaging, 
and  provided  bounds  on  the  time  of  termination  for  the  proposed  scheme.  Next,  we 
investigated  an  estimator-based  control  strategy  for  a  scenario  in  which  the  agents 
have  a  process  noise  in  addition  to  the  decentralized  information  structure. 

The  rest  of  the  paper  is  organized  as  follows.  Section  2  presents  the  jamming  and 
communication  models  for  the  nodes.  Based  on  the  aforementioned  models,  a  multi¬ 
player  pursuit-evasion  game  is  analyzed  in  Section  3  between  an  aerial  jammer  and  a 
pair  of  UAVs.  Section  4  generalizes  the  jamming  problem  introduced  in  the  previous 
section  to  address  the  scenario  when  there  is  an  arbitrary  number  of  agents /players 
in  the  network.  Section  5  presents  an  approximation  scheme  for  the  players  based 
on  a  decentralized  information  structure.  In  Section  6,  we  analyze  a  discrete-time 
version  of  the  jamming  problem  with  process  noise  and  decentralized  information 
structure,  and  propose  an  estimator-based  strategy  for  the  players.  Finally,  we 
conclude  in  Section  7  with  some  current  challenges  and  future  research. 


2.  Jamming  model.  In  this  section,  we  first  introduce  a  communication  model 
between  two  mobile  nodes  in  the  presence  of  a  jammer.  Consider  a  mobile  node 
{receiver)  receiving  messages  from  another  mobile  node  {transmitter)  at  some  fre¬ 
quency.  Both  communicating  nodes  are  assumed  to  be  lying  on  a  plane.  Consider 
a  third  node  that  is  attempting  to  jam  the  communication  channel  in  between  the 
transmitter  and  the  receiver  by  sending  a  high  power  noise  at  the  same  frequency. 
This  kind  of  jamming  is  referred  to  as  the  trivial  jamming.  A  variety  of  metrics  can 
be  used  to  compare  the  effectiveness  of  various  jamming  attacks.  Some  of  these  met¬ 
rics  are  energy  efficiency,  low  probability  of  detection,  and  strong  denial  of  service 
[50]  [48].  In  this  paper,  we  use  the  ratio  of  the  jamming-power  to  the  signal-power 
(JSR)  as  the  metric.  From  [53],  we  have  the  following  models  for  the  JSR  {£)  at 
the  receiver’s  antenna. 


.  ^  PJtGjrGrj  ^  „log^^(:^) 

PtGtrGrt 


1.  i?”  model 
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2.  Ground  Reflection  Propagation 

PjtGjrGrj  /  Dtr\^ 

PtGtrGrt  V/iT’/  \Djii) 

3.  Nicholson 

.  ^  PJtGjrGrj  41ogio(:^) 

PtGtrGrt 

where  is  the  power  of  the  jammer  transmitting  antenna,  Pt  is  the  power  of 
the  transmitter,  Gtr  is  the  antenna  gain  from  transmitter  to  receiver,  Grt  is  the 
antenna  gain  from  receiver  to  transmitter,  Gjr  is  the  antenna  gain  from  jammer 
to  receiver,  Grj  is  the  antenna  gain  from  receiver  to  jammer,  hj  is  the  height  of 
the  jammer  antenna  above  the  ground,  Ht  is  the  height  of  the  transmitter  antenna 
above  the  ground,  Dtr  is  the  Euclidean  distance  between  transmitter  and  receiver, 
and  Dju  is  the  Euclidean  distance  between  jammer  and  receiver.  All  the  above 
models  are  based  on  the  propagation  loss  depending  on  the  distance  of  the  jammer 
and  the  transmitter  from  the  receiver.  In  all  the  above  models  the  jammer  to  signal 
ratio  is  dependent  on  the  ratio  Dtr/ Dju. 

For  digital  signals,  the  jammer’s  goal  is  to  raise  the  ratio  to  a  level  such  that  the 
hit  error  rate  [54]  is  above  a  certain  threshold.  For  analog  voice  communication, 
the  goal  is  to  reduce  the  articulation  performance  so  that  the  signals  are  difficult  to 
understand.  Hence  we  assume  that  the  communication  channel  between  a  receiver 
and  a  transmitter  is  considered  to  be  jammed  in  the  presence  of  a  jammer  if  ^  >  ^tr 
where  ^tr  is  a  threshold  determined  by  many  factors  including  application  scenario 
and  communication  hardware.  If  all  the  parameters  except  the  mutual  distances 
between  the  jammer,  transmitter  and  receiver  are  kept  constant,  we  can  conclude 
the  following  from  all  the  above  models: 

If  Djr/ Dtr  <  r]  for  some  positive  constant  rj,  then  the  communication  channel 
between  a  transmitter  and  a  receiver  is  considered  to  be  jammed. 

Here  ry  is  a  function  of  Pj^.,  Pt,  Gtr,  Grt,  Gjr,  Grj  and  Dtr-  Hence  if 
the  transmitter  is  not  within  a  disc  of  radius  Djr/p  centered  around  the  receiver, 
then  the  communication  channel  is  considered  to  be  jammed.  We  call  this  disc  as 
the  perception  range.  The  perception  range  for  any  node  depends  on  the  distance 
between  the  jammer  and  the  node.  For  effective  communication  between  two  nodes, 
each  node  should  be  able  to  transmit  as  well  as  receive  messages  from  the  other 
node.  Hence  two  nodes  can  communicate  if  they  lie  in  each  other’s  perception  range. 

In  the  rest  of  the  paper,  we  will  use  the  above  jamming  and  communication 
model. 

3.  Aerial  jamming  in  a  UAV  communication  network.  In  this  section,  we 
envision  a  scenario  in  which  an  aerial  jammer  intrudes  the  communication  link  be¬ 
tween  two  UAVs.  In  the  past  few  years,  a  lot  of  research  has  been  devoted  to 
deploy  multiple  UAVs  in  a  decentralized  manner  to  carry  out  tasks  in  military  as 
well  as  civilian  scenarios.  In  general,  the  mode  of  communication  among  UAVs 
deployed  in  a  team  mission  is  wireless.  This  renders  the  communication  chan¬ 
nel  vulnerable  to  malicious  attacks  from  aerial  jammer  flying  in  their  vicinity.  In 
the  past  year,  there  have  been  reports  of  predator  drones  being  hacked  [24,  43], 
resulting  in  intruders  gaining  access  to  classified  data  being  transmitted  from  an 
aircraft.  Motivated  by  such  incidents,  our  first  scenario  addresses  the  problem  of 
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evading  a  jamming  attack  launched  by  an  aerial  jammer  flying  in  the  vicinity  of 
two  UAVs  that  are  trying  to  mutually  communicate  among  themselves.  We  model 
the  intrusion  as  a  continuous  time  pursuit-evasion  game  between  the  UAVs  and 
the  aerial  jammer.  Such  dynamic  games  governed  by  differential  equations  can  be 
analyzed  using  tools  from  differential  game  theory  [40]  [27].  In  the  past,  differential 
game  theory  has  been  used  as  a  framework  to  analyze  problems  in  multi-player 
pursuit-evasion  games.  Solutions  for  particular  multi-player  games  were  presented 
by  Pashkov  and  Terekhov  [52] ,  Levchenkov  and  Pashkov  [39] ,  Hagedorn  and  Break- 
well  [26],  Breakwell  and  Hagedorn  [17]  and  Sriram  et.a/.[56].  More  general  treatment 
of  multiplayer  differential  games  was  presented  by  Starr  and  Ho  [16],  Leitmann  [36], 
Vaisbord  and  Zhukovskiy  [65],  Zhukovskiy  and  Salukvadze  [70]  and  Stipanovic,  Ho- 
vakimyan  and  Melikyan  [62].  The  inherent  hardness  in  obtaining  an  analytical 
solution  to  Hamilton- Jacobi-Isaacs  equation  has  led  to  the  development  of  numer¬ 
ical  techniques  for  the  computation  of  the  value  function.  Recent  efforts  in  this 
direction  to  compute  an  approximation  of  the  reachable  sets  have  been  provided 
by  Mitchell  and  Tomlin  [47],  Stipanovic,  Hwang  and  Tomlin  [61]  and  Stipanovic, 
Sriram  and  Tomlin  [60].  In  contrast  to  the  previous  work  in  pursuit-evasion  games 
that  formulate  a  payoff  based  on  a  geometric  quantity  in  the  configuration  space  of 
the  system,  we  formulate  a  payoff  based  on  the  capability  of  the  players  in  a  team 
to  communicate  among  themselves  in  the  presence  of  a  jammer  in  their  vicinity.  In 
particular,  we  are  interested  in  computing  strategies  for  spatial  reconfiguration  of 
a  formation  of  UAVs  in  the  presence  of  an  aerial  jammer  to  reduce  the  jamming  on 
the  communication  channel. 

In  the  following  subsection,  we  describe  the  kinematic  model  of  the  UAVs. 


3.1.  System  model.  In  our  analysis,  each  node  is  a  UAV.  We  consider  two  UAVs 
(UAVi  and  UAV2)  in  the  presence  of  a  third  UAV  (UAVj)  that  is  trying  to  jam 
the  communication  link  in  between  them.  We  assume  that  the  UAVs  are  having  a 
constant  altitude  flight.  This  assumption  helps  to  simplify  our  analysis  to  a  planar 
case.  Referring  to  Figure  8,  the  configuration  of  each  UAV  in  the  global  coordinate 
frame  can  be  expressed  in  terms  of  the  variables  (xf,  y®,  The  subscript  i  is 
either  1,  2  or  j  depending  on  the  UAV  being  referred.  The  pair  represents 

the  position  of  a  reference  point  on  UAVi  with  respect  to  the  origin  of  the  global 
reference  frame  and  denotes  the  instantaneous  heading  of  the  UAVi  in  the  global 
reference  frame.  Hence  the  state  space  for  UAVi  is  Xi  =  x  S^.  In  our  analysis, 
we  assume  that  the  UAVs  are  a  kinematic  system  and  hence  the  dynamics  of  the 
UAVs  are  not  taken  into  account  in  the  differential  equation  governing  the  evolution 
of  the  system.  The  kinematics  of  the  UAVs  are  assumed  to  be  the  following: 


dt 


Wi  cos  (j)f ; 


dt 


WiSin(/)f;  ^  =  Ui, 


(1) 


where  Wi  and  Ci  are  the  speed  and  angular  velocity  of  UAVi,  respectively.  In  this 
paper,  we  assume  that  ai  G  [—1,  -1-1]  Vi.  Moreover,  we  assume  that  Wi  =  1  Vi. 

The  state  space  of  the  entire  system  is  Xi  x  X2  x  Xj  =  R®  x  (S^)^.  In  order  to 
reduce  the  dimension  of  the  state  space  we  analyze  the  system  in  a  coordinate  frame 
fixed  to  UAV2  as  shown  in  Figure  8.  In  the  new  coordinate  frame,  the  system  can  be 
modeled  using  six  independent  variables  and  the  equations  of  motion  of  the  UAVi 
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Figure  1 .  Configuration  of  a  UAV 


and  UAVj  with  respect  to  the  new  coordinate  frame  are  given  by  the  following  [56] : 


f  xi  \ 
yi 

X2 

y2 

\h  ) 


/  -1  +  (T2?/1  +  cos  \ 
—(72X1  +  sin  01 
—  (72  +  CTl 

—l  +  (72yj+  COS0J 

—a2Xj  +  sin  0j 
V  -<72  +  (7j  J 

V 

/(x,<Ti.cr2,crj) 


(2) 


In  the  above  expressions  (a;j,yj,0j)  and  (a;i,yi,0i)  represent  the  relative  position 
and  orientation  of  the  UAV^  and  UAVi  in  the  reference  frame  attached  to  UAV2 
which  are  the  state  variables  of  the  system.  Hence  the  state  space  of  the  reduced 
system  is  isomorphic  to  x  (S^)^. 


Figure  2.  Relative  configuration  of  UAVs 
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3.2.  Problem  statement.  From  the  communication  and  the  mobility  models  pro¬ 
posed  in  the  previous  subsections,  we  formulate  the  following  problem.  Consider  a 
situation  in  which  UAVi  and  UAV2  are  not  communicating  initially  in  the  presence 
of  a  jammer  (UAVj).  The  objective  of  the  jammer  is  to  maximize  the  time  for  which 
it  can  jam  the  communication  between  UAVi  and  UAV2.  The  objective  of  UAVi 
and  UAV2  is  to  minimize  the  time  for  which  communication  remains  jammed.  The 
game  terminates  at  the  first  instant  at  which  UAVi  and  UAV2  are  in  a  position 
to  communicate.  We  need  to  compute  the  optimal  strategies  for  each  UAV.  We 
assume  that  each  UAV  has  a  complete  knowledge  about  the  state  of  the  system. 

3.3.  Analysis.  We  consider  a  situation  in  which  UAVi  and  UAV2  are  not  commu¬ 
nicating  initially  in  the  presence  of  a  jammer  (UAVj).  The  termination  condition  is 
defined  as  the  hrst  instant  at  which  UAVi  and  UAV2  are  in  a  position  to  commu¬ 
nicate.  The  cost  function  of  the  game  is  the  time  of  termination  of  the  game.  The 
objective  of  the  jammer  is  to  maximize  the  time  for  which  it  can  jam  the  commu¬ 
nication  between  UAVi  and  UAV2.  The  objective  of  UAVi  and  UAV2  collectively 
is  to  minimize  the  time  for  which  communication  remains  jammed. 

In  order  to  obtain  the  optimal  strategies  of  the  players  we  need  to  compute 
the  saddle-point  strategies  since  this  is  a  zero-sum  game.  A  set  of  strategies  for 
the  players  are  said  to  be  in  saddle-point  equilibrium  if  no  unilateral  deviation  in 
strategy  by  a  player  can  lead  to  a  better  outcome  for  that  player.  Hence  there  is  no 
motivation  for  the  players  to  deviate  from  their  equilibrium  strategies.  In  scenarios 
where  the  players  have  no  knowledge  about  each  other’s  strategies,  equilibrium 
strategies  are  important  since  they  lead  to  a  guaranteed  minimum  outcome  for  the 
players  in  spite  of  the  other  player’s  strategies. 

For  a  point  x  in  the  state  space,  let  J (x)  represent  the  outcome  if  the  players  im¬ 
plement  their  optimal  strategies  starting  at  the  point  x.  In  this  game,  it  is  the  time 
of  termination  of  the  game  when  the  players  implement  their  optimal  strategies.  It  is 
also  called  the  wa/ite  of  the  game  at  X.  LetVJ=[Ja;i  Jyi  Jyj 

denote  the  gradient  of  the  value  function.  The  Hamiltonian  of  the  system  is  given 
by  H  =  1  -I- V  J  -  /(x,  crj',  (T*,  cr|,  t).  From  the  equations  of  motion  of  the  system,  the 
Hamiltonian  is  given  by  the  following  expression: 


H  —  1  -j-  T  JyiVi  T  T  U  Jy^Vj  ~t- 


Since  the  jammer  wants  to  maximize  the  time  of  termination  and  the  UAV’s  want  to 
minimize  the  time  of  termination,  we  get  the  following  expressions  for  the  controls 
from  Isaacs’  first  condition. 


(cr*,  cr2i  =  arg  max  mini? 

^  /T„-  /To/Ti 


(7  j  2  <7 


Since  the  Hamiltonian  is  separable  in  its  controls,  the  order  of  taking  the  extrema 
becomes  inconsequential.  Hence  the  optimal  control  of  the  players  are  given  as 
follows. 


JyjXj  -\-  JxjUj] 

a*  =  sign(j0j 
CTi  =  -sign(J,^J 


(3) 

(4) 

(5) 
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The  retrogressive  path  equations  (RPE)  for  the  system  lead  to  the  following  equa¬ 
tions. 


Jxi  —  ^2'^yi')  Jy\  — 

Jxj  —  ’^yj  ~  ^2'^Xj 

J(p-Y  —  Jx\  SilKpl  Jy-Y  COS  01 

J(f)j  —  sin  Jyj  cos 

°  denotes  derivative  with  respect  to  retrograde  time. 


(6) 

(7) 

(8) 
(9) 


Figure  3.  The  Control  Loop  for  the  System 

Figure  3  summarizes  the  entire  control  algorithm.  The  controller  of  each  UAV 
takes  as  input  the  state  variables  and  runs  the  RPE  to  compute  the  control.  This 
control  is  then  fed  into  the  plant  of  the  respective  UAV.  The  plant  updates  the  state 
variables  based  on  the  kinematic  equations  governing  the  UAV.  Finally  the  sensors 
feedback  the  state  variables  into  the  controllers.  In  this  case  the  sensors  measure 
the  position  and  the  orientation  of  each  UAV. 

3.4.  Termination  situations.  In  order  to  compute  the  optimal  strategies,  we  need 
to  compute  the  boundary  conditions  for  the  dependent  variables  of  the  differential 
equation.  In  order  to  do  so,  we  characterize  the  terminal  conditions  of  the  game 
in  the  state  space  and  compute  the  value  of  VJ  at  the  terminal  conditions.  This 
section  presents  the  computation  of  the  terminal  value  of  the  dependent  variables 
of  the  differential  equations  governing  the  game. 

From  the  communication  model,  we  can  conclude  that  UAVi  can  communicate 
with  UAV2  when  the  following  condition  holds: 

r;min[d(UAVj,UAVi),d(UAVj,UAV2)]  >  d(UAVi,UAV2) 

where  c?(UAVi,UAVj)  is  the  Euclidean  distance  between  UAVi  and  UAV^.  Hence 
the  boundary  of  the  game  set  is  the  set  of  positions  of  the  UAV’s  that  satisfies  the 
following  condition: 

r;min[d(UAVj,UAVi),d(UAVj,UAV2)]  =  d(UAVi,UAV2) 

This  leads  to  two  termination  manifolds  in  the  state  space. 
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1.  The  first  terminal  manifold  is  characterized  by  the  positions  of  the  UAV’s  such 
that  UAVi  is  at  the  boundary  of  the  perception  range  of  UAV2  and  UAV2  is 
inside  the  pereeption  range  of  UAVi .  In  the  coordinate  system  of  UAV2  the 
terminal  manifold  is  represented  by  the  hypersurface  Fi{xi,yi,(j)i,Xj,yj,(l)j) 
which  is  given  by  the  following  expression: 


{\Jxl  +  Vi-ri\Jx]  +  y]  =  0)  n 

{{xi  -  Xjf  +  (yi  -  yjf  -  [x]  +  y])  <  0) 

2.  The  second  terminal  manifold  is  characterized  by  the  positions  of  the  UAV’s 
such  that  UAV2  is  at  the  boundary  of  the  perception  range  of  UAVi  and 
UAVi  is  inside  the  perception  range  of  UAV2.  In  the  coordinate  system  at¬ 
tached  to  UAV2  this  terminal  manifold  is  represented  by  the  hypersurface 
F2 (xi ,yi,(j)i,  Xj ,  yj ,  (pj )  is  given  by  the  following  expression: 

{\Jxl+yl-T]\Jx‘]  +  y^  =  0)  n 

((ti  -  Xjf  +  (yi  -  yjf  -x‘^j+  y]  >  0) 


Both  the  terminal  surfaces  are  five  dimensional  manifolds  with  boundary.  Hence 
they  can  be  parameterized  using  five  independent  variables  xi,yi,Xj,(j)i  and  (pj. 
Since  J  =  0  on  the  terminal  manifold,  VJ  satisfies  the  following  equations  at  an 
interior  point  in  the  manifold: 


tO  I  tO 

dxi 


I  tO 

dxj 


=  0, 


tO  I  jO 

dyi 


=  0 


=  0,  J?=0,  JI.  =0 


0 


(10) 


In  addition  to  the  above  equations  Isaacs’  second  condition  leads  to  the  following 
equation. 


iJ(x,VJ,/(x,at,a2*,a;))  =  0  (11) 

The  value  of  V  J  at  the  terminal  manifold  can  be  obtained  from  Equations  (11) 
and  (12).  Since  there  are  two  different  terminal  manifolds,  we  have  to  analyze  both 
of  them  separately.  At  first,  we  compute  the  value  of  VJ  on  terminal  manifold  1. 
Substituting  the  expression  for  Fi{xi,yi,(l)i,Xj,yj,(l)j)  in  Equation  (11)  and  (12), 
we  obtain  the  following  value  of  Jy^  . 


Jl  =  +  (,?)-(-  -  1)  +  (x?  - 


1)1 


-1 


(12) 


From  the  values  of  VJ,  the  optimal  control  for  the  players  and  their  higher 
derivatives  at  termination  are  given  as  follows: 

a*,  =  -sign(J«J 


J. 


0 

01 


=  0 
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Figure  4.  Figure  shows  the  players  leading  to  Termination  con¬ 
dition  1  for  Problem  1.  The  value  r/  =  1.  The  player  in  red  is 
the  jammer.  The  players  in  green  and  blue  are  UAVi  and  UAV2 
respectively.  Figure  (b)  shows  the  control  of  the  UAVi.  Figure  (c) 
shows  the  control  of  the  UAVj.  Figure  (c)  shows  the  control  of  the 
UAV2. 
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Figure  5.  Figure  shows  the  players  leading  to  Termination  con¬ 
dition  2  for  Problem  1.  The  value  rj  =  2.  The  player  in  red  is 
the  jammer.  The  players  in  green  and  blue  are  UAVi  and  UAV2 
respectively.  Figure  (b)  shows  the  control  of  the  UAVi.  Figure  (c) 
shows  the  control  of  the  UAV,/.  Figure  (c)  shows  the  control  of  the 
UAV2. 


T° 


(4- 


-1 


=  criCi(x°) 


(13) 
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a*  =  sign(J^.) 

4.  =  0 

J%,=0 

+  (!/?)‘(^  - 1)  + 


(^2  —  sign[(/3;^yi  Jy-^X\  J(f>\ 

{JxiVl  Jyi^l  ^(pi  '^(pj  ^XjUj^  —  0 

(*^3:1^1  Jyx^l  J(p\  ’^fpj  '^Vj^j  ^XjUj^  — 


(14) 


(y°  -  +  (m^  -i)+(^°-$)r‘  (15) 

From  equation  (9)  we  can  conclude  the  following 

.  >  0  ^  4  <  0  ^  jo^  >  0  ^  ajf  <  0  ^  ci(x°)  <  0 

.  Jl<0^jl  >0^Jl  <0^  at  >0^  ci(x°)  <  0 
From  equation  (10)  we  can  conclude  the  following 

•  4j  ^  0  ^  4j  ^  0  ^  4j  ^  ^  ^  4  >  0  c j  (x®)  >  0 

•  4j  ^  0  ^  4j  >  0  <  0  cr *  <  0  =>  c j  (x°)  >  0 

From  the  expressions  of  at  and  a* ,  we  can  conclude  that  sign(ci  (x° ) )  =  sign(cj  (x° ) ) . 
This  implies  that  if  at  termination  ci(x°)  <  0  then  =  0  =>  cr*  =0  and  if 
Cj  (x°)  <  0  then  =  0  CTi  =  0. 

Repeating  the  same  analysis  at  the  second  terminal  manifold  leads  to  the  follow¬ 
ing  values  for  the  and  controls  at  termination. 


70 


=  („o  -  ,;)|  VEEBi!E  +  +  (,.)!+ 

y(i»F+feS)i)(~s(*?  -  -A")  - 1)  -  ^)i“‘ 


a 


* 

1 


=  -sign(j0j 

4.  =  o 


4i  =  [y^  cos  (j)i  -  Xj  sin  4>i]p{x°) 
a*  =  sign(  ) 

Jl  =  0 


=  [Vj  cos()!)i  -  x°  sin0i]p(x°) 
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•  (Jn 


a'r 


^  <f)i  ^4’j  '^yj'^3  '  ^XjUj ) 

(  JO  7,0  _  70  ^0  _  7-0  _  70  _  70  ^0  I  70  7,0^1  _ 
^yi-^l  ^(Pj  ^yj^j  ^  ^XjUj )  — 

yi  /  0\ 

- X 

T]^ 


where  p(x°)  =  .  +  ^(x°)2  +  (?/0)2(cos((/)?  -  (/)°)  -  1)  -  ^)]  ^ 


Figure  6.  Figure  shows  the  players  leading  to  Termination  con¬ 
dition  1  for  Problem  2.  The  value  77  =  2.  The  player  in  red  is 
the  jammer.  The  players  in  green  and  blue  are  UAVi  and  UAV2 
respectively.  Figure  (b)  shows  the  control  of  the  UAVi.  Figure  (c) 
shows  the  control  of  the  UAV,/.  Figure  (c)  shows  the  control  of  the 
UAV2. 


3.5.  Results.  Figures  4,  5,  6  and  7  show  trajectories  of  the  players  along  with 
their  optimal  controls  for  various  terminal  conditions  and  different  values  of  77.  The 
position  of  the  players  corresponding  to  the  termination  situation  is  shown  by  a 
small  circle  in  the  plots  showing  the  trajectories  of  the  players.  Each  figure  shows 
the  trajectory  of  the  players  just  before  termination  for  a  small  time  interval.  From 
the  expression  of  the  optimal  controls  in  equations  (4),  (5)  and  (6),  we  can  infer  that 
the  controls  of  the  players  are  bang-bang.  This  is  also  verified  from  the  simulation 
results.  From  the  nature  of  the  controls  and  kinematics  of  the  system,  we  can  infer 
that  the  optimal  paths  comprise  of  arcs  of  circles  and  straight  line  trajectories  as 
motion  primitives.  Arcs  of  circles  are  generated  when  the  UAV  keeps  its  angular 
velocity  saturated  at  one  extrema  for  a  non-zero  interval  of  time.  Straight  line 
segments  are  obtained  due  to  rapid  switching  between  the  extremal  value  of  the 
controls  (chattering).  An  instance  of  such  a  behavior  is  exhibited  by  UAV2  in 
Figure  4. 
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Figure  7.  Figure  shows  the  players  leading  to  Termination  con¬ 
dition  2  for  Problem  2.  The  value  77  =  1.  The  player  in  red  is 
the  jammer.  The  players  in  green  and  blue  are  UAVi  and  UAV2 
respectively.  Figure  (b)  shows  the  control  of  the  UAVi.  Figure  (c) 
shows  the  control  of  the  UAV,/.  Figure  (c)  shows  the  control  of  the 
UAV2. 


4.  Connectivity  maintenance  in  the  presence  of  an  adversary.  Next,  we 
investigate  the  problem  of  jamming  in  a  mobile  network  having  an  arbitrary  num¬ 
ber  of  agents,  and  analyze  it  from  the  perspective  of  maintaining  connectivity  in  a 
network  of  mobile  agents  in  the  presence  of  a  mobile  intruder.  Substantial  research 
has  been  done  in  the  recent  past  to  address  the  problem  of  maintaining  connectiv¬ 
ity  among  autonomous  agents.  Based  on  tools  from  potential  field  methods  and 
algebraic  graph  theory,  centralized  algorithms  have  been  proposed  in  [68]  and  [66] 
to  maintain  connectivity  in  mobile  networks.  The  authors  use  the  dynamics  of  the 
Laplacian  matrix  in  order  to  obtain  feasible  controls  that  maintain  connectivity  in 
addition  to  satisfying  the  differential  constraints  on  the  motion  of  each  agent.  In 
[58],  the  notion  of  geometric  connectivity  robustness  is  introduced  as  a  measure  of 
the  local  connectedness  of  a  network.  Furthermore,  the  authors  show  that  under 
special  conditions  the  new  notion  provides  a  sufficient  condition  for  global  connectiv¬ 
ity  of  the  network.  In  [31],  the  authors  use  the  weighted  graph  Laplacian  technique 
proposed  in  [30]  to  guarantee  connectivity  while  achieving  formation  stabilization. 
In  [21],  a  decentralized  algorithm  is  presented  for  maintaining  connectivity  using 
the  Laplacian  of  the  proximity  graph.  In  [23],  the  problem  of  maintaining  connec¬ 
tivity  is  addressed  for  agents  having  second-order  dynamics.  The  authors  establish 
an  existence  theorem  for  connectivity  maintenance  and  present  optimal  controls 
to  maintain  connectivity  in  a  distributed  fashion.  In  [67],  [69],  the  authors  pro¬ 
pose  a  distributed  feedback  and  provably  correct  control  framework  for  connectivity 
maintenance  in  addition  to  accounting  for  communication  delays  as  well  as  collision 
avoidance.  In  [32],  verification  for  the  correctness  of  the  asynchronous  and  parallel 
computation  proposed  in  [67]  is  provided  by  experimental  analysis  on  a  team  of 
robots.  Most  of  the  prior  work  deals  with  the  problem  of  maintaining  connectivity 
due  to  the  distributed  architecture  of  sensing  and  communication  in  multi-agent 
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systems,  which  provides  increased  efficiency,  performance,  scaiabiiity  and  robust¬ 
ness.  In  contradistinction,  this  section  focuses  on  maintaining  connectivity  for  a 
network  within  an  arbitrary  number  of  mobiie  agents  in  the  presence  of  a  mobiie 
jammer  in  the  vicinity. 

We  assume  that  there  are  m  agents  in  the  network  in  the  presence  of  a  jammer. 
Let  the  dynamics  associated  with  the  ith  agent  be  given  by  the  foiiowing  equation: 

ii=fiixi,Ui)  (16) 

where,  Xi  G  m  G  Ut  {(j)  :  [0,t]  — >■  Ai  \  (j){-)  is  measurable},  where  Ai  C 
fi  :  H.”*  X  >  R  is  uniformly  continuous,  bounded  and  Lipschitz  continuous  in  Xi 
for  fixed  Ui.  Consequently,  given  a  fixed  iti(-)  and  initial  point,  there  exists  a  unique 
trajectory  solving  (16)  [1].  Let  the  state  of  node  i  be  denoted  as  Xi  G 'Ki  C 

Let  X,  denote  the  state-space  of  the  jammer.  We  assume  that  the  jammer  has 
the  following  dynamics  associated  with  itself: 

x,  =  f,{x,,u,)  (17) 

where  x,  G  M"*,  u,  G  U,  {<j)  :  [0,t]  — ^  A,  \  </>(•)  is  measurable},  where  A,  C 
.  /,  :  R”*  X  R  is  uniformly  continuous,  bounded  and  Lipschitz  continuous 

in  X,  for  fixed  it,. 

Let  X  =  Xi  X  •  •  •  X  Xm  X  X,  C  0^  R"*  x  R"*  represent  the  entire  state  of  the 
system,  where,  0  represents  the  Cartesian  product  of  the  Euclidean  spaces  R^L 
Let  u  =  [it}’  •  •  •  it})]^  be  a  column  vector  that  represents  the  control  of  all  the  agents 
in  the  network. 

We  define  the  workspace  [19]  as  the  ambient  space  in  which  the  agents  exist. 
Since  we  are  interested  in  vehicular  networks,  the  ambient  space  of  the  nodes  is 
either  R^  or  R^ .  Since  all  the  agents  reside  in  the  same  ambient  space,  we  use  to 
denote  the  workspace  for  all  agents.  Let  Xi  denote  the  coordinates  of  the  ith  agent 
in  n.  We  assume  that  is  equipped  with  a  distance  metric  p  :  fl  x  fl  — >■  R. 

As  a  simple  example  to  highlight  the  difference  between  the  state  space  and 
the  workspace,  consider  the  following  second  order  agent  that  can  only  move  in  a 
straight  line  with  it  as  the  control  input. 

Xi  =  X2 
±2  =  U 

The  state  space  for  the  system  can  be  represented  by  the  vector  x  =  [xi  X2Y' ■ 
The  state  space  is  two  dimensional  but  the  agent  can  only  move  on  a  straight  line 
and  hence  the  workspace  is  fl  =  R^.  For  any  state  x,  x  =  xi.  For  two  states 
x*,x**  G  fl,  p{x\x**)  =1  x}  -x}*  |. 

Based  on  the  above  definitions,  the  network  connectivity  maintenance  problem 
is  formulated  as  the  following  zero-sum  differential  game  between  the  jammer  and 
the  nodes  in  the  network. 

Gl'.  Consider  a  situation  in  which  the  network  is  initially  eonneeted  in  the  pres¬ 
ence  of  a  jammer.  The  instantaneous  states  of  all  the  agents  and  the  jammer  are 
known  to  all  the  players  till  termination  of  the  game.  The  ohjeetive  of  the  jammer 
is  to  minimize  the  time  in  which  it  can  disconnect  the  communication  network  by 
jamming  the  communieation  ehannel  between  agents.  The  objective  of  agents  is  to 
maximize  the  time  for  which  communication  link  between  them  remains  operable. 
The  game  terminates  at  the  first  instant  at  which  the  agents  are  about  to  lose  their 
links.  We  need  to  compute  the  optimal  strategies  for  each  agent.  Disconnection 
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refers  to  a  situation  in  which  there  are  agents  i  and  j  such  that  there  is  no  path  in 
the  communication  network  to  transmit  messages  between  them. 

As  in  the  previous  section,  we  want  to  compute  the  saddle-point  strategies  of  the 
players  in  the  game.  In  [12],  we  have  analysed  the  scenario  of  multiple  jammers  and 
multiple  UAVs  flying  in  a  formation.  We  analyzed  the  problem  in  the  framework 
of  differential  game  theory,  and  provided  analytical  and  approximate  techniques  to 
compute  non-singular  motion  strategies  of  the  UAVs.  Analytical  solutions  to  the 
associated  Hamilton-Jacobi-Bellman-Isaacs  equations  becomes  extremely  difficult 
to  compute  as  the  number  of  players.  Moreover,  the  non-linearity  present  in  the 
dynamical  equations  of  the  players  makes  it  difficult  to  obtain  an  analytical  solution 
to  the  optimal  control  problem  addressed  by  the  Isaacs’  conditions.  Many  computa¬ 
tional  techniques  have  been  proposed  to  compute  the  optimal  trajectories  for  such 
problems  but  they  are  computationally  intensive  even  for  systems  evolving  in  low 
dimensions  [47],  [61],  [60]. 

Due  to  the  inherent  difficulty  in  solving  the  above  differential  game,  we  propose  a 
greedy  strategy  for  the  agents.  In  the  following  sections,  we  model  the  connectivity 
of  the  network  as  a  graph,  and  formulate  a  cost  function  based  on  it. 

4.1.  State-dependent  graphs.  In  this  section,  we  model  the  underlying  commu¬ 
nication  network  within  the  agents  as  a  graph.  The  connectivity  of  the  network 
depends  on  the  position  of  the  agents  relative  to  the  jammer.  Since  the  agents 
and  the  jammer  are  assumed  to  be  mobile,  the  connectivity  of  the  network  evolves 
in  time  rendering  the  graph  to  be  dynamic  in  nature.  Moreover,  topology  of  the 
graph  depends  on  the  state  of  the  nodes  and  therefore,  we  can  use  the  framework 
of  state-dependent  graphs  introduced  in  [46]  to  map  the  state  of  the  system  to  a 
graph.  A  state-dependent  graph  is  a  mapping,  g^,  from  the  state  space  X,  to  the 
set  of  all  labeled  graphs  on  m  vertices,  G{m),  i.e., 

:  X  G{m) 

Node  i  in  the  graph  represents  agent  i.  It  is  assumed  that  the  order  of  these  graphs 
at  all  times  is  m  since  the  number  of  agents  is  independent  of  time.  Let  E{gc{x)) 
denote  the  edge-set  of  the  graph  under  consideration.  Now  we  specify  how  the 
existence  of  a  communication  link  dictates  the  existence  of  an  edge  between  a  pair 
of  vertices  in  the  state-dependent  graph  G.  For  nodes  i  and  j  with  states  Xi  G  X^ 
and  Xj  G  Xj  respectively,  we  consider  the  subset  Sij  C  X^x  Xj  to  define  the  edge 
between  i  and  j  if  the  following  condition  is  satisfied: 

ij  G  E{gc{x))  if  and  only  if  (xi,Xj)  G  Sij  (18) 

The  jamming  model  proposed  in  Section  II  leads  to  the  following  definition  of  Sij. 
Let  d  =  p(xi,Xj),  where  Xi  and  Xj  are  the  coordinates  of  the  nodes  i  and  j  in  the 
workspace  O  equipped  with  a  distance  metric  p.  Let  Br[p\  =  {y  G  U  j  p{y,p)  <  r}. 
From  the  above  discussion  we  can  conclude  the  following: 

Sij  —  i.i.Xi^Xj')  ]  ^  7Ijyc/[x2]  U  7Ijyc/[Xj]}  (19) 

The  above  statement  along  with  (18)  means  that  if  the  jammer  lies  within  a  distance 
rjd  from  either  of  the  nodes  then  the  communication  channel  is  assumed  to  be 
jammed.  The  collection  of  edge  states  is  denoted  as 

S  —  with  Sij  d  Xj  x  X^ 
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From  [46],  the  state  dependent  graph  is  defined  as  follows: 


Definition:  Given  the  set  system  5,  the  map  :X— >■  Gm  with  an  image  consisting 
of  graphs  of  order  m,  having  an  edge  between  vertex  i  and  j  iff  {xi,Xj)  €  Sij  is 
defined  as  a  state-dependent  graph  with  respect  to  S. 

Now  that  we  have  a  mapping  gc  from  the  state  of  the  system  to  a  graph  on  m 
vertices,  we  can  study  the  properties  of  the  graphs  based  on  the  state  of  the  system. 


4.2.  Dynamic  networks.  In  this  section,  we  present  control  strategies  for  con¬ 
nectivity  maintenance  based  on  the  algebraic  properties  of  graphs  that  quantify  the 
connectivity  of  the  underlying  communication  network.  In  order  to  do  so,  we  need 
to  define  the  following  mathematical  objects  associated  with  a  graph  G  having  m 
nodes: 


1.  Adjacency  matrix  :  It  is  an  to  x  to  matrix  with  entries  given  as  follows: 


J  I  if  an  edge  exists  between  i  and  j 
^  0  if  no  edge  exists  between  i  and  j 


2. 


Laplacian  of  a  graph  (£(G))  :  It  is  an  to  x  to  matrix  with  entries  given  as 
follows: 


(a)  aij 


—  1  if  an  edge  exists  between  i  and  j 
0  if  no  edge  exists  between  i  and  j 


(b)  an  —  '^k=i,k^i 


In  a  dynamic  network,  since  G  is  a  function  of  xsX,  its  adjacency  matrix  is  also 
a  function  of  the  state  x.  Let  A(x)  denote  the  adjacency  matrix  of  the  graph  G. 
The  element  Oij  =  I  if  an  edge  exists  between  nodes  i  and  j  or  else  it  is  zero 
=  1  *j(f  (xi,Xj)  €  Sij.  Let  di  =  p{x,,Xi),  dj  =  p{x,,Xj)  and  dij  =  p(xi,Xj). 

Ghanges  in  A(x)  occur  at  discrete  points  in  time.  On  the  other  hand,  the 
dynamics  of  the  nodes  and  the  jammer  are  continuous  in  time.  In  order  to  relate 
the  discrete  nature  of  A(x)  to  the  continuous-time  dynamics  of  the  nodes  we  use 
the  following  continuous  approximation  for  aij : 


aij(x)  =  u{di  -  rjdij)  ■  u{dj  -  gdij) 


where  u{-)  is  a  continuous  approximation  to  the  Heavyside  step  function  given  by 
the  following  logistic  function: 


u{y) 


I 

I  -f  e-^y 


As  limfc_>.oo,  the  logistic  function  takes  the  following  form: 


u(y) 


1  y>0 

0  otherwise 


Hence  k  can  be  used  as  a  parameter  to  vary  the  rate  at  which  the  exponential 
function  decays  in  the  neighborhood  of  zero.  The  dynamics  of  the  ay  (x)  can  be 
written  as  follows: 


dij  (x)  =  VxOy  (x)  •  X  (20) 

where  Vxay  (x)  denotes  the  mn  x  1  vector  which  is  the  gradient  of  aij(x)  w.r.t  x. 

The  second-smallest  eigenvalue  of  £(G)  is  called  the  Fiedler  value,  denoted  as 
X2{C{G)).  It  is  also  called  the  algebraic  connectivity  of  G.  It  has  emerged  as  an 
important  parameter  in  many  systems  problems  defined  over  networks.  In  [22],  [49], 
[29],  it  has  also  been  shown  to  be  a  measure  of  the  stability  and  robustness  of  the 


PURSUIT-EVASION  GAMES 


17 


networked  dynamic  system.  Since  this  work  deals  with  connectivity  maintenance 
in  the  presence  of  a  malicious  intruder  X2{C{G))  arises  as  a  natural  parameter  of 
interest  for  both  players. 

For  a  graph  G  to  be  connected,  X2{£{G))  >  0  [14].  Therefore,  in  order  to 
maintain  connectivity,  the  nodes  in  the  network  must  move  in  the  presence  of  a 
jammer  so  as  to  satisfy  the  above  condition.  On  the  other  hand,  the  jammer  must 
move  in  a  way  to  make  A2(/1(G))  =  0.  In  the  remaining  part  of  this  section,  we 
assume  that  the  network  is  initially  connected. 

From  the  above  discussion  a  control  law  can  be  designed  for  the  nodes  so  as 
to  keep  A2 (/C(G))  a  non-decreasing  function  of  time  9{\2{c{G)))  ^  Since 

is  also  a  function  of  the  controls  of  the  jammer  it  might  not  be  possible 
for  the  nodes  to  satisfy  the  above  condition  at  all  times.  Instead  the  following 
objective  leads  to  a  feasible  control  for  the  nodes  at  all  times: 

(21) 

dt  ^  ^ 

On  the  other  hand,  the  jammer  must  move  in  a  way  so  as  to  make  A2(G(G))  = 
0.  Therefore,  a  plausible  strategy  for  the  jammer  is  keep  A2(G(G))  a  decreasing 
function  at  all  times.  As  in  the  previous  case  such  an  objective  might  not  lead  to  a 
feasible  control  strategy  at  all  times.  Therefore,  the  jammer  can  have  the  following 
objective  in  order  to  yield  a  feasible  control  at  all  times: 

Minimize  :  — ^  ^2  0  (22) 

Since  C{G)  is  a  symmetric  positive  semi-definite  matrix,  all  its  eigenvalues  are  non¬ 
negative.  Therefore  the  jammer  cannot  decrease  A2(G(G))  once  it  reaches  0.  This 
leads  to  the  additional  constraint  in  its  objective. 

In  order  to  satisfy  the  above  objective  for  the  players  we  need  a  relation  between 
the  control  of  the  agents  and  Since  A2(£(G))  is  a  function  of  the  relative 

positions  of  the  agents  in  a  network  we  can  get  a  relation  between  A2(£(G))  and 
the  Ui-  From  [28],  we  obtain  the  following  expression: 


dX2(£(G)) 

d£ 


V2V2 

VIV2 


(23) 


where  V2  is  the  eigenvector  corresponding  to  the  A2(G(G)). 

Consider  agent  i  having  state  Xi  G  Let  xt  =  [x'f^,---  ,a:-"‘^]'^.  Let  fi  = 

We  can  use  the  chain  rule  to  obtain  the  following  expression: 


dX2{£{x))  _  dX2{£)  d£ 


(24) 


where,  {A,B)  =  tr{A^B),  an  inner  product  for  the  space  of  matrices.  Hence  we 
obtain  the  following  relation  between  control  Ui  of  each  agent: 


5A2(£(G)) 

dt 


IfL  11% 

EE( 


dX2{£) 

d£ 


d£ 


dx. 


(fc) 
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Therefore,  a  locally  optimal  control  law  for  the  agents  is  a  solution  of  the  follow¬ 
ing  optimization  problem: 


1.  Node  i:  u*  =  max„,  ^ 


2.  Jammer:  it*  =  min„.  TW 


In  the  next  section,  we  present  some  simulations  based  on  the  above  control  law 
for  the  agents. 


4.3.  Results.  We  consider  a  network  of  agents  moving  in  a  plane  in  the  vicinity  of 
a  jammer.  All  the  agents,  including  the  jammer,  are  holonomic  kinematic  agents 
with  fixed  speeds.  The  differential  equation  governing  the  motion  of  agent  i  is  as 
follows: 


Xi  =  Ui  cos  6i 
Ui  =  Ui  sin 

The  differential  equation  governing  the  motion  of  the  jammer  is  as  follows: 

Xm  =  It,  cos  9, 
y,  =  It,  sin  9, 


Using  the  control  laws  from  the  previous  section,  we  obtain  the  following  controls 
for  the  agents  and  the  jammer: 

1.  Node  V. 


(cos  0i,sin  9i) 


5A2(£)  dL  d\2{C)  dL 
dL  '  dxL^^  dL  '  dyr 


2.  Jammer: 


(cos  0,,sin  0,) 


dX2{L)  dL  dX2{L)  dL 
dL  '  dxj'^  dL  '  dyj’ 


Figures  8  and  9  show  simulations  for  20  agents  (m  =  20)  for  which  the  above 
control  scheme  is  implemented.  In  Figure  8,  the  speed  of  all  the  agents  is  same 
as  that  of  the  jammer.  In  Figure  9,  the  speed  of  tth  agent  is  it,  -|-  —  0.5. 

The  simulations  are  run  till  the  jammer  succeeds  in  disconnecting  the  network  for 
the  first  time.  Figures  8(a)  and  9(a)  show  the  Frobenius  norm  of  the  error  in 
approximating  the  Laplacian  matrix  as  a  function  of  time.  Figures  8(b)  and  9(b) 
show  the  variation  of  the  Fiedler  value  with  time. 

Until  now,  we  assumed  that  at  each  instant  all  the  agents  had  a  complete  knowl¬ 
edge  about  the  entire  state  of  the  system.  In  order  for  this  to  be  possible,  each  agent 
has  to  continuosly  communicate  its  state  information  to  all  the  other  agents.  This 
can  lead  to  a  congestion  in  the  communication  network  of  the  formation  leading  to 
unwanted  delay  in  information  dissemination.  In  the  following  sections,  we  present 
strategies  for  each  agent  based  on  limited  state  information  about  other  agents  in 
the  network. 


5.  Decentralized  decision  making  due  to  communication  constraints.  In 

this  section,  we  investigate  decentralized  techniques  for  maintaining  connectivity 
among  agents  in  the  presence  of  a  mobile  jammer  in  the  vicinity.  In  the  past,  deriva¬ 
tion  of  distributed  algorithms  for  the  computation  of  non-cooperative  equilibria  of 
certain  classes  of  games  has  been  addressed  in  [4,  41].  It  has  been  shown  in  these 
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papers  that  when  the  cost  function  is  convex  or  if  the  system  is  linear  with  quadratic 
cost,  then  the  policy  iteration  algorithm  converges  to  the  Nash  equilibrium.  In  the 
absence  of  centralized  information,  decentralized  decision  making  has  been  proven 
to  be  intractable  for  discrete  static  team  decision  problems  [51,  64,  3].  Some  ap¬ 
proximation  schemes  are  available  for  discrete  team  decision  problems  under  limited 
information  [20].  However,  there  has  been  limited  work  on  solving  differential  game 
problems  under  limited  information  [18].  In  this  section,  we  consider  a  scenario  in 
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which  each  agent  makes  a  decision  based  on  a  limited  information  available  to  it 
regarding  other  agents  in  the  network.  Additionally,  we  assume  that  each  agent 
has  a  knowledge  about  the  value  function  of  the  game  addressed  in  the  previous 
section,  under  perfect  state  information,  beforehand.  Due  to  lack  of  information 
about  all  the  agents  in  the  team  each  agent  is  constrained  to  make  a  local  deci¬ 
sion.  We  propose  a  decentralized  algorithm  for  each  agent,  and  provide  a  bound 
on  its  performance  compared  to  the  optimal  solution  obtained  under  a  centralized 
information  pattern. 

We  investigate  two  different  approximation  schemes  for  the  agents.  In  the  first 
scheme,  we  assume  that  each  agent  has  a  knowledge  about  the  solution  of  the  game 
under  the  assumption  that  the  information  pattern  is  centralized.  Based  on  this 
solution,  the  players  device  averaging  schemes  in  order  to  approximate  their  optimal 
control  actions. 

In  the  previous  section,  we  introduced  the  following  connectivity  maintenance 
game: 

Gl:  Consider  a  situation  in  which  the  network  is  initially  connected  in  the  pres¬ 
ence  of  a  jammer.  The  instantaneous  states  of  all  the  agents  and  the  jammer  are 
known  to  all  the  players  till  termination  of  the  game.  The  objective  of  the  jammer 
is  to  minimize  the  time  in  which  it  can  disconnect  the  communication  network  by 
jamming  the  communication  channel  between  agents.  The  objective  of  agents  is  to 
maximize  the  time  for  which  communication  link  between  them  remains  operable. 
The  game  terminates  at  the  first  instant  at  which  the  agents  are  about  to  lose  their 
links.  We  need  to  compute  the  optimal  strategies  for  each  agent.  Disconnection 
refers  to  a  situation  in  which  there  are  agents  i  and  j  such  that  there  is  no  path  in 
the  communication  network  to  transmit  messages  between  them. 

In  the  present  work,  we  make  the  following  assumptions: 

1.  We  assume  that  all  the  agents  have  an  a  priori  knowledge  of  the  value  of  the 
game  GI.  We  refer  to  this  as  the  value  map  since  the  techniques  proposed  in 
this  work  require  every  agent  to  use  this  information  for  navigation.  The  value 
at  a  state  x  €  X  is  denoted  as  J(x).  In  addition,  we  assume  the  dynamics 
of  the  agents  to  be  decoupled.  Moreover,  each  agent  computes  its  optimal 
control  using  the  following  expression: 

u*(x(t))  =  argmax(l  -k  Jx^  •  Mx,,Ui))  (25) 

Ui 

2.  We  assume  that  agent  i  broadcasts  Xi{t)  at  all  times. 

3.  Due  to  the  presence  of  a  jammer  in  the  vicinity,  an  agent  might  lack  state 
information  about  the  other  agents  as  the  game  proceeds.  Therefore,  we 
assume  that  each  agent  has  state  information  only  regarding  those  agents 
from  which  it  receives  the  broadcast  signals. 

4.  The  jammer  executes  its  optimal  control  since  it  is  assumed  that  the  jammer 
has  complete  knowledge  of  X(t)  at  all  times. 

From  the  last  assumption,  we  can  infer  that  an  agent  i  has  an  accurate  knowl¬ 
edge  of  Xj{t)  iff  i  lies  in  the  perception  range  of  agent  j  at  time  t.  There¬ 
fore,  at  time  t,  the  state  information  available  to  each  agent  can  be  expressed 
as  a  directed  graph,  I{t)  =  {V,E{t))  where,  V  =  {vi  ■  ■  ■  Vm,Vm}  and  E{t)  = 
{viVj\j  is  in  the  perception  range  of  i\.  Since  the  agents  as  well  as  the  jammer  are 
mobile,  /  is  a  function  of  t  and  therefore,  a  dynamic  graph  [45].  We  define  the 
neighborhood  of  i  in  /  as  N}{t)  =  {j\vjh^i  G  E{t)}. 
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All  the  aforementioned  assumptions  are  motivated  from  our  previous  works  in 
[10,  11,  9,  8]  regarding  jamming  in  formation  of  UAVs  and  multi-agent  systems.  In 
the  following  subsection,  we  propose  an  approximation  scheme  under  a  decentral¬ 
ized  information  pattern  that  uses  the  value  map  for  the  agents  to  formulate  their 
optimal  control  laws  in  such  scenarios. 

5.1.  Approximation  scheme.  In  this  section,  we  introduce  an  approximation 
scheme  for  agents  to  compute  their  controls.  These  control  laws  are  based  on  the 
value  of  the  game  under  full-state  information  structure. 

If  each  agent  had  a  perfect  knowledge  of  X(t)  then  it  could  compute  u*  (t)  using 
(25).  Due  to  the  presence  of  a  jammer,  the  state  information  available  to  agent  i  is 
restricted  to  Nj{t).  Based  on  this  information,  at  time  t,  the  system  can  lie  on  any 
point  on  a  manifold,  Mjyj,  given  by  the  following  expression; 

={xGX|Afc  =  Afc(t),  VkeN}(t)} 

For  each  x  G  agent  i  can  compute  u*(x)  using  (25)  and  the  value  map. 

Since  the  agent  i  lacks  perfect  information  about  X(t)  on  Adjyi,  it  cannot  compute 
<(x(f)). 

Now  we  present  a  decentralized  approximation  scheme  that  uses  averaging  in 
order  to  compute  the  control  law  for  each  agent.  In  this  strategy,  agent  i  computes 
the  average  of  the  vector  u*(x)  over  the  entire  manifold  in  order  to  obtain  its 

control.  The  average,  Ui,  on  is  given  by  the  following  expression: 

.  u*(x)-dx 

7 - ^  (2') 

We  assume  that  G  U{t).  This  restricts  the  models  of  the  agents  for  which 
our  analysis  is  applicable.  From  the  definition  of  we  can  see  that  it  is  a 

submanifold  of  Moreover,  since  we  assume  that  X(0)  is  known  and 

the  maximum  control  of  each  agent  is  bounded,  Adjv]  can  be  estimated  to  be  a 
bounded  and  compact  submanifold.  Therefore,  we  can  perform  the  integration 
in  the  numerator  by  assuming  u*  (x)  to  be  an  appropriate  n-form  on  the  charts 
comprising  the  atlas  chosen  on  In  the  simplest  case  when  Adjv)  Euclidean, 

the  n-form  is  given  by  c?x  =  dx^  •••da:'"".  The  denominator  is  also  called  the 
volume  of  the  submanifold.  In  the  simplest  case  when  is  2-dimensional,  it  is 

the  area  of  the  submanifold  embedded  in  the  higher  dimensional  space.  An  inherent 
assumption  made  in  the  above  definition  is  that  i®  orientable.  In  case  Adjyi 

is  not  Euclidean,  the  integration  can  performed  over  singular  chains  [59]. 

From  an  initial  state  x  G  X,  the  time  of  termination  of  the  game  when  all  the 
agents  follow  the  approximation  scheme  is  denoted  as  T (x) . 

Theorem  5.1.  If  following  assumptions  hold: 

1-  WJxih  <  a,  \/xi 

2.  |iti*(a:)||  <  §,  Vi 

3.  f{xi,  Ui)  is  lipschitz  in  Ui  with  lipschitz  constant  K  for  all  Xi. 

4-  Aijg-i  is  a  Euclidean  manifold 

Then  the  termination  time  of  the  game  obeys  the  following  condition: 

Jixo)  <  T{xo)  < 


X 


M 


(27) 
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where  5  =  mKaj3. 


Proof.  The  rate  of  change  of  the  value  function  at  time  t  along  the  trajectory 
traced  when  the  players  follow  the  approximation  scheme  is  given  by  the  following 
expression: 

iapp(t)  =  ^  Jxi(t)  •  /(x*(t),  Uj)  +  ■  /(x,(t),  u,*)  (28) 

In  the  above  expression,  the  jammer  uses  its  optimal  control  since  it  can  be  com¬ 
puted  from  the  value  map  due  to  the  assumption  that  the  jammer  has  a  complete 
state  information  at  all  times.  From  the  definition  of  the  value  of  the  game  and  u*, 
the  rate  of  change  of  J  at  time  t  is  given  by  the  following  expression: 

jit)=  ■ /(x*(t),u*) • /(x,(t),u.*)  (29) 


Let  us  define  e  =  Japp  —  J-  From  (28)  and  (29),  we  conclude  the  following: 
e{t)  =  ■  [/(x*,Ui)  - /(xi,u*)] 

|e|  <  IkxJb  •  |l/(x*,u*)  - /(x„u*)||2 

<  K  Y.  ll-^x.lb-llu.-Klb 


Let  F(x)  = 


M, 


,m] 

=  K  Y 

,m] 

1. 


Im  <  «(y)-<W)'^y 


I 


=  ^  E 


«(y)  -  <(x))dy|| 


Therefore,  using  the  third  assumption  we  can  obtain  the  following  bound: 

UxJ|2  f 


E 

,m]  ''  ' 


F(x) 


IM, 


lluHa;)  -  u*{y)\\2dy 


From  the  second  assumption  on  bounds  on  the  control  law,  we  can  conclude  that 
l|u*(x)-u*(y)||2  <  /3.  Therefore,  the  above  expression  is  bounded  by  the  following: 

\e\<Kp  Y  (30) 

,m] 


From  the  assumptions  at  the  beginning  of  this  section,  we  obtain  the  following 
bound  on  the  error  rate: 


|e|  <  maf3K  =  6  (31) 

The  optimal  control  laws  for  the  agents  maximize  the  Hamiltonian,  which  is 
(1  -I-  J)  in  this  game.  Therefore,  the  following  always  holds 

japp<j  e  <  0  (32) 

From  the  definition  of  e,  we  obtain  the  following  relation: 

japp(t)  >  j(t)  -  S 
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Integrating  the  above  equation  on  both  sides  from  t  =  0  to  t  =  T (x)  we  obtain  the 
following: 


rT 


J app  dt  ^ 


^app(T(xo))  -  T(Xo)  > 


r 


Jdt  — 


T 


Sdt 


Jo  Jo 

j(r(xo))  -  j(xo)  -  5r(xo) 


Japp{T{xo  j)  =  0,  since  the  network  is  disconnected  at  termination.  J(T(xo))  >  0, 
since  the  termination  time  of  the  original  game  is  greater  than  T(xo).  Therefore, 
we  obtain  the  following  bound  for  T(xo): 

r(xo)  <  (33) 

From  the  above  expression,  we  get  an  upperbound  for  7^(xo).  The  lower  bound  is 
trivially  obtained  from  the  definition  of  T(xo).  □ 


In  the  next  section,  we  address  a  special  case  in  which  the  dynamics  of  each  agent 
is  linear,  and  includes  a  process  noise. 


6.  Estimator  design  techniques.  In  this  section,  we  underscore  the  difficulty 
associated  with  the  formulation  of  a  general  multi-agent  networked  control  system 
with  process  noise  which  is  under  attack.  An  important  issue  that  arises  in  the  case 
of  control  with  process  noise  under  noisy  observation  is  that  of  certainty  equiva¬ 
lence  [57,  55,  2,  3,  6].  Even  though  certainty  equivalence  is  a  well-known  property 
of  the  optimal  controller  in  the  linear-quadratic-Gaussian  (LQG)  problem,  it  has 
been  shown  that  this  does  not  readily  carry  over  to  similarly  structured  differential 
or  dynamic  games  [2] .  Even  in  zero-sum  games  with  identical  (noisy)  measurements 
for  the  two  players,  only  a  restricted  version  of  certainty  equivalence  holds  [55,  3]. 
No  general  theory  exists  that  would  be  applicable  to  the  type  of  problems  considered 
in  this  paper.  Moreover,  earlier  studies  on  decentralized  control  systems  have  shown 
that  only  under  specific  assumptions  on  the  state  transition  function,  control  law 
and  noise  processes  affecting  the  system,  a  closed-form  solution  is  achievable.  Since 
obtaining  optimum  the  solution  is  a  formidable  task,  we  resort  to  an  approximation 
scheme,  which  is  based  on  designing  estimators  for  the  agents  that  can  be  imple¬ 
mented  during  the  actual  play  of  the  game.  In  addition  to  the  assumptions  made  in 
the  problem  considered  in  the  previous  section,  we  make  the  following  assumptions 
in  this  section  : 

•  Each  agent  as  well  as  the  jammer  knows  the  initial  state  of  all  the  agents  and 
the  jammer. 

•  We  assume  that  each  agent  and  the  jammer  has  a  linear  control  law  (this  is 
possible,  if  the  search  for  optimal  control  law  is  restricted  to  linear  control 
laws  for  all  agents  and  the  jammer  while  computing  the  value  function  for  the 
game) . 

•  We  assume  that  a  process  noise  is  acting  on  each  agent  and  the  jammer  during 
the  course  of  the  game. 

Under  these  assumption,  we  obtain  the  dynamics  of  the  error  in  the  estimate  of 
each  agent  at  each  time  instant.  This  problem  is  considerably  more  difficult  than 
the  problems  considered  in  the  previous  section,  since  we  do  not  know  a  priori  if 
certainty  equivalence  holds  in  this  case. 
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6.1.  Discrete-time  system.  We  slightly  deviate  from  the  notations  used  in  the 
previous  section.  Consider  m  agents  and  a  jammer,  each  governed  by  the  following 
discrete-time  linear  equation 

{t  +  1)  =  A,x\t)  +  Biu'"  (t)  +  w\t) 

for  z  =  1, 2, . . . ,  m,  m  -I-  1,  t  =  0, 1,  2, ...,  N. 


where,  x^{t)  G  R"  is  the  state,  u*(t)  G  is  the  control,  G  R"  is  the  zero-mean 
process  noise  with  bounded  support  for  the  agent.  The  systems  matrices  for 
agent  are  Ai  G  R”^”  and  Bi  G  Each  agent  is  assumed  to  observe  its  state 

perfectly.  We  assume  that  each  agent  has  a  linear  control  law  zz®(t)  =  Kix{t)  where 
Ki  G  Rpx’T-l’Ti+i)  is  the  gain  and  x(t)  is  a  column  vector  comprised  of  states  of 

all  agents  stacked  together,  i.e.,  x(t)  =  [(a;^(t))’^, . . . ,  The  jammer, 

assumed  to  be  the  m+  1*^  agent,  chooses  control  actions  to  disconnect  the  network 
graph  that  leads  to  an  intermittent  state  information  available  to  the  agents  about 
the  states  of  the  other  agents. 

Recall  that  I(t)  is  the  communication  graph  of  the  agents.  Let  us  now  construct 
a  communication  matrix  ^(/(t))  :=  i,j  =  1,2, ...,  (m  +  1)^  as  follows 


%m) 


1  Onxn 

i-1 

m+1 

[  ^nxn 

otherwise. 

-I- 1,  mod(z,  771-1-1 


eE{t) 


where  Inxn  is  an  identity  matrix  of  dimension  n  x  n  and  0„xn  is  a  zero  matrix 
of  dimension  n  x  n.  One  can  notice  that  the  entries  in  this  matrix  have  a  strong 
coupling  with  the  spatial  arrangement  of  the  agents  and  their  distance  with  the 
jammer. 

Let  Xi{t)  G  R"('"+^)  denote  the  estimated  state  of  all  agents  and  Ui(t)  G  RpI^+i) 
denote  the  estimated  control  of  all  agents  based  on  the  estimate  of  the  state  Xi(t)  by 
agent  z.  It  should  be  noted  that  at  each  point  in  time,  agent  i  has  information  from 
his  neighbors  and  its  estimate  Xi(t)  contains  the  exact  information  about  his 

neighbors  and  himself.  Let  us  denote  the  error  in  the  estimate  Xi{t)  of  each  agent 
from  the  actual  state  x(t)  by  ei(t),  i.e. 

Biit)  =x{t) -Xi[t).  (35) 

The  error  values  eiit)  change  during  the  course  of  the  game  due  to  the  distur¬ 
bances  on  some  agents  and/or  non-optimal  behavior  of  some  agents,  as  discussed 
in  the  formulation.  By  stacking  each  ei(t),  we  get  £{t)  :=  [ef’ (t), . . .  G 

R"’(™+i)  We  have  the  following  lemma  about  the  dynamics  of  the  error  £{t). 
Before  the  lemma,  we  introduce  some  notations  that  are  used  in  the  proof  of  the 
lemma. 


6.1.1.  Preliminary  notation  for  Lemma  6.1.  Let  1  G  1  :=  [1, 1, ... ,  1]"^  and 

e  (g)  1  :=  [e^, . . . ,  be  the  usual  Kronecker  product  of  matrices.  Let  w(t)  := 
[(7«^(t))^, . . . ,  .  Let  us  dehne  the  following  matrices: 

•  A  :=  diag[Ai,  A2, ...,  Am+i]  G  R"(”^+i)x™(m-Hi) 

•  B  =  diag[Bi,B2,...,Bm+i]  G 

•  K  =  diag[Ai,  A2,...,  A„+i]  G  Rp(’"+i)xn(m-Hi)=^ 
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Also,  define  A  €  such  that  A  =  [Aij]  for  z,  j  €  {1,  2, m  + 1}, 

and  B  e  M’^(™+i)xn(m+i) 


=  (A +  B) 


^n(i—l)xn(m+l) 

B,K, 

On(m+l— -i)  xn(m+l) 


II 

0n(j  — 1)  xn(m+l) 
-BjK, 

,  B  = 

(  BiKi  \ 

B2K2 

\  0n(m-|-l— j)xn(77i-|-l)  ) 

Lemma  6.1.  The  vector  £(t)  of  errors  in  the  estimates  of  all  agents  grows  as 
£{t  +  1)  =  {^[I{t  +  l))A)£{t)  +  w{t)  0  1,  f  (0)  =  0. 


(36) 


Proof.  Since  each  agent  is  running  an  estimator  to  construct  its  strategy,  they  are 
actually  using  a  control,  say  v(t),  which  is  different  from  their  optimal  strategy, 
u*{t),  under  perfect  state  information.  The  approximate  control  v(t)  is  constructed 
by  stacking  the  control  policy  of  each  agent,  obtained  using  their  approximate  states 
from  their  respective  estimators.  The  estimated  control  Ui(t)  is  based  on  the  state 
estimate  ii{t)  of  the  i*^  agent. 


/  AiXi(t)  \ 

/  KiSci{t)  \ 

K2^2{t) 

K2%{t) 

v(t)  = 

Ui(0  = 

\  j 

The  difference  between  v(t)  and  Uift)  is 


V(t)  -  Ui{t) 


(  Ki{{yi{f)  -  ±^{t))  -  {yi{t)  -  yii{t)))  \ 
K2lix{t)  -  5Ci{t))  -  {Kit)  -  X2(t))) 

\  Arm+l((x(t)  -  Xi(t))  -  (x(t)  -  X™+i(t)))  / 

/  Ki{ei(t)  -  ei{t))  \ 

K2  {ei{t)  -  62  (t)) 

\  (ej (t)  em_|_i(t))  j 


The  state  x(t)  and  the  estimator  Xi(t)  for  agent  i  evolve  according  to 


x(t  +  1)  =  Ax(t)  +  Bv(<)  +  ’w(t) 

Xi(t  +  1)  =  Axi(t)  +  Bui(t),  z  =  1,  2, ...,  m  +  1. 


Now,  we  study  the  growth  of  error  Giif).  The  equation  for  error  dynamics  is  given 

by 


ei(t  +  l)  =  Aei(t) +B(v(t)  -  Ui(t)) +w(t), 

=  (A  +  S)e,(<)-BKf(t)+w(t). 

Let  us  now  consider  the  matrix  of  all  errors  £{t).  Since  agent  z  will  receive  the 
state  information  from  its  neighbors  in  the  connectivity  graph  I(t  +  1)  at  time  t  + 1, 
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some  entries  in  its  error  vector  ei(t  +  1)  will  be  zero.  The  dynamics  of  errors  can 
be  combined  into  one  equation  of  the  form 

£{t  +  1)  =  +  l))A)£{t)  +  w(f)  (g)  1. 

The  initial  condition  for  the  error  dynamics  is  £1(0)  =  0,  since  the  state  information 
of  all  the  agents  and  the  jammer  are  available  to  each  other  at  the  beginning  of  the 
game.  □ 

The  error  in  the  estimate  of  the  state  (t)  depends  on  the  error  in  the  estimate 
of  the  other  agents.  Therefore,  an  agent  cannot  make  any  judgment  about  the 
accuracy  of  his  own  estimate  Xi(t).  However,  stability  of  error  can  be  inferred  if 
the  gradient  of  value  function  J  is  available  and  initial  state  is  known.  If  the  value 
function  is  obtained  numerically,  then  the  stability  of  the  error  can  be  established 
using  numerical  tools. 

The  error  dynamics  given  in  equation  (36)  are  driven  by  i.i.d.  noise.  Because  of 
the  presence  of  noise,  asymptotic  convergence  of  error  to  zero  is  not  possible.  In 
order  to  ensure  bounded  error  throughout  the  horizon,  the  error  dynamics  must  be 
stable.  For  discrete  time  linear  systems,  this  is  equivalent  to  the  spectral  radius 
A  being  less  than  1.  However,  the  dynamics  here  is  that  of  a  switched  system  for 
which  the  stability  conditions  are  different. 

From  the  Lyapunov  stability  criterion  for  switched  systems  [42],  we  know  that 
the  error  values  remain  bounded  if  we  can  hnd  a  positive  definite  matrix  P,  such 
that  the  following  holds  true 

+  l))w4)^  P  -\-  P  +  l))w4)  ^  —aln(m+l)‘^  xn(m+l)‘^  ^  (37) 

for  some  positive  a  at  all  time  t.  This,  however,  is  not  applicable  to  the  problem 
at  hand.  Since  many  agents  remain  connected  in  the  system,  some  of  the  diagonal 
entries  in  +  1))  remain  0,  which  in  turn  implies  that  +  1))A)^  P  + 

P{‘^{I{t  +  l))w4)  leaves  some  rows  with  zeros  as  all  its  elements.  Hence,  the  left 
hand  side  cannot  be  negative  definite  at  any  time  t. 

We  can  notice  that  the  control  strategy  of  the  agents  of  the  game  play  dual 
role  in  the  system.  One,  which  is  obvious,  is  that  they  are  trying  to  optimize 
certain  objective  functional  and  achieve  some  goal  (which  in  this  case  is  to  maintain 
connectivity).  The  other,  which  is  hidden  is  that  they  are  also  acting  on  the  error 
dynamics  of  the  system.  The  control  policy  can  be  carefully  chosen  to  achieve 
optimal  trade-off  between  these  two  seemingly  unrelated  (maybe  even  conflicting) 
goals. 

7.  Conclusion.  In  this  paper,  we  considered  the  problem  of  jamming  in  a  commu¬ 
nication  network  within  a  team  of  autonomous  mobile  agents.  First,  we  considered 
a  differential  game-theoretic  approach  to  compute  optimal  strategies  for  a  team 
of  UAVs  trying  to  evade  a  jamming  attack  initiated  by  an  aerial  jammer  in  their 
vicinity.  We  formulated  the  problem  as  a  zero-sum  pursuit-evasion  game,  where 
the  cost  function  is  the  termination  time  of  the  game.  We  used  Isaacs’  approach 
to  obtain  necessary  conditions  to  arrive  at  the  equations  governing  the  saddle-point 
strategies  of  the  players.  We  illustrated  the  results  through  simulations.  Next,  we 
analyzed  the  problem  of  jamming  from  the  perspective  of  maintaining  connectivity 
in  a  network  of  mobile  agents  in  the  presence  of  an  adversary.  This  is  a  variation 
of  the  standard  connectivity  maintenance  problem  in  which  the  main  issue  is  to 
deal  with  the  limitations  in  communications  and  sensing  model  of  each  agent.  In 
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our  work,  the  limitations  in  communication  are  due  to  the  presence  of  a  jammer 
in  the  vicinity  of  the  mobile  agents.  We  computed  evasion  strategies  for  the  team 
of  vehicles  based  on  the  connectivity  of  the  resultant  state- dependent  graph.  We 
presented  some  simulations  to  validate  the  proposed  control  scheme.  Finally,  we 
addressed  the  problem  of  jamming  for  the  scenario  in  which  each  agent  computes 
its  control  strategy  based  on  limited  information  available  about  its  neighbors  in 
the  network.  Under  this  decentralized  information  structure,  we  proposed  two  ap¬ 
proximation  schemes  for  the  agents  and  studied  the  performance  of  the  entire  team 
for  each  scheme. 

Among  the  future  works  are  to  extend  the  locally  optimal  trajectories  presented 
in  this  paper  into  the  entire  phase  space  so  as  to  obtain  the  globally  optimal  tra¬ 
jectories.  This  is  an  extremely  hard  problem  due  to  the  curse  of  dimensionality 
associated  with  the  HJI  equations.  Moreover,  it  would  require  the  construction  of 
various  types  of  singular  surfaces  [44,  5,  40].  Other  interesting  directions  are  to 
incorporate  communication  delays  within  the  agents,  and  address  the  problem  of 
inter-agent  collision  avoidance  among  the  autonomous  agents. 
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